Skip to content

feat: rename repo surface and add source-aware Databricks targets#21

Merged
anand-testcompare merged 2 commits into
mainfrom
feat/convex-sync-kit-source-aware-databricks
Apr 21, 2026
Merged

feat: rename repo surface and add source-aware Databricks targets#21
anand-testcompare merged 2 commits into
mainfrom
feat/convex-sync-kit-source-aware-databricks

Conversation

@anand-testcompare

@anand-testcompare anand-testcompare commented Apr 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • rename the repo-facing surface from convex-streaming-olap-export to convex-sync-kit
  • switch release automation to a repo-wide release-please contract so fixes outside apps/convex-sync still open the stable release PR
  • add a source-config layer with sources/meshix-api/env.sh and make the Databricks S3/Delta scripts source-aware
  • add a real Delta bootstrap entrypoint and align Databricks production naming to convex_sync_kit_meshix_api_{s3,delta_control,delta_bronze,delta_silver}

Verification

Local

  • cargo fmt --all --check
  • cargo test --workspace
  • python -m py_compile platform/databricks/delta/extractor/convex_cdc_job.py
  • bash -n scripts/load-source-config.sh scripts/bootstrap-databricks-delta.sh scripts/ensure-databricks-delta-secret.sh scripts/deploy-databricks-delta.sh scripts/run-databricks-delta-job.sh scripts/run-databricks-delta-smoke.sh scripts/sync-databricks-staging-views.sh install.sh scripts/convex-sync-dev
  • actionlint -config-file .github/actionlint.yaml .github/workflows/*.yml .depot/workflows/*.yml
  • DATABRICKS_BUNDLE_ENGINE=direct databricks bundle validate -p DEFAULT -t prod ... from platform/databricks/delta
  • npx --yes release-please manifest-pr --repo-url=shpitdev/convex-sync-kit --target-branch=feat/convex-sync-kit-source-aware-databricks ... --dry-run --debug
    • result: would open chore(feat/convex-sync-kit-source-aware-databricks): release 0.0.3

Live Databricks

  • ./scripts/bootstrap-databricks-delta.sh DEFAULT 63d28889f3eb3c4b
    • created workspace.convex_sync_kit_meshix_api_delta_{control,bronze,silver}
  • ./scripts/deploy-databricks-delta.sh DEFAULT prod
  • ./scripts/run-databricks-delta-job.sh DEFAULT prod
    • created/ran convex-sync-kit-meshix-api-prod-delta-extract
    • run 170978464304345 finished SUCCESS
  • checkpoint verification:
    • workspace.convex_sync_kit_meshix_api_delta_control.connector_checkpoint_latest
    • latest row: meshix-api / delta_tail
  • bronze verification:
    • workspace.convex_sync_kit_meshix_api_delta_bronze
    • SHOW TABLES returned 21 CDC tables
  • S3-backed Databricks rename:
    • ./scripts/sync-databricks-staging-views.sh --warehouse-id 63d28889f3eb3c4b --bucket shpitdev-convex-streaming-olap-export-628031635965 --prefix prod --label meshix-api-prod
    • created workspace.convex_sync_kit_meshix_api_s3 with 30 views
    • verified __source_map
    • deleted old workspace.convex_streaming_olap_export

Open in Devin Review

devin-ai-integration[bot]

This comment was marked as resolved.

@anand-testcompare anand-testcompare merged commit 6df4cf0 into main Apr 21, 2026
9 checks passed
@anand-testcompare anand-testcompare deleted the feat/convex-sync-kit-source-aware-databricks branch April 21, 2026 06:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant